An Adaptive Computational Discourse System based on Data-driven Learning Algorithm

This paper suggests a system which takes the role of café clerks and communicates with customers. The automatic dialog system is based on an algorithm of machine learning to estimate the intention of customers’ talks which are entered into the system as texts and generate appropriate clerks’ replies. Since the system learns characteristics of dialogues in the café from a corpus collected from real cafés, it is able to communicate with people naturally in café situations and it is also applicable to other circumstances. The suggested system consists of four modules: a module which extracts information such as the menu from an input sentence, one which inserts the extracted information into an output sentence, one which guesses the intention of entered customers’ request and one which produces an output based on the estimated customers’ intention and the collected conversation corpus. The modules which obtain additional information from the input and place it into the output are implemented to use the prior information like the menu because this study limits the experiment to conditions in the café. The others are designed on the basis of the Hidden Markov Model and a method of filtering and prediction. To do the experiment, we recorded approximately 130 dialogues of ordering situations in a café and 12 dialogues each from another café and a bus terminal. Through processes of converting conversations into text and tagging the intention of each talk, the collected conversations were used for learning both the intention estimating module, a next sentence generating module and testing their performances. Two types of the performance tests were conducted to evaluate the accuracy of the algorithm to estimate customers’ intention and suitability of the generated clerk’s words. 10-fold cross validation with 130 ordering dialogues were used for the accuracy evaluation of the intention estimation; the data set was divided into 10 subsets randomly, and 9 subsets were used for the module’s learning while one subset was utilized for the test. The averaged accuracy showed that the module is able to guess the intention of the input sentence with 91 percent accuracy. Meanwhile, the naturalness of generated sentences was tested with a 7point scale. The surveyors rated each sentence with 5.18 points and each dialogue (one set of conversation) with 4.47 points on average. Additionally, the system which used 130 dialogues for training was applied to cases of dialogues from another café and a bus terminal, and it showed the possibility of application in conditions other than a café by gaining 5.88 points for each sentence and 5.95 points for each dialogue on average in the situation of the bus terminal whose scores were higher than that of other café; 5.32 points for each sentence and 4.45 for each dialogue.